Using healthcare systems data for outcomes in clinical trials: issues to consider at the design stage

Background Healthcare system data (HSD) are increasingly used in clinical trials, augmenting or replacing traditional methods of collecting outcome data. This study, PRIMORANT, set out to identify, in the UK context, issues to be considered before the decision to use HSD for outcome data in a clinical trial is finalised, a methodological question prioritised by the clinical trials community. Methods The PRIMORANT study had three phases. First, an initial workshop was held to scope the issues faced by trialists when considering whether to use HSDs for trial outcomes. Second, a consultation exercise was undertaken with clinical trials unit (CTU) staff, trialists, methodologists, clinicians, funding panels and data providers. Third, a final discussion workshop was held, at which the results of the consultation were fed back, case studies presented, and issues considered in small breakout groups. Results Key topics included in the consultation process were the validity of outcome data, timeliness of data capture, internal pilots, data-sharing, practical issues, and decision-making. A majority of consultation respondents (n = 78, 95%) considered the development of guidance for trialists to be feasible. Guidance was developed following the discussion workshop, for the five broad areas of terminology, feasibility, internal pilots, onward data sharing, and data archiving. Conclusions We provide guidance to inform decisions about whether or not to use HSDs for outcomes, and if so, to assist trialists in working with registries and other HSD providers to improve the design and delivery of trials. Supplementary Information The online version contains supplementary material available at 10.1186/s13063-024-07926-z.

The MHRA guidance states 'It is recommended that an observational feasibility study is conducted by the sponsor to assess the recent capture of study variables prior to undertaking a RWD study' and 'It is expected that the validity of the RWD that are intended to be used in the study is formally documented and approved by the sponsor before the study protocol is published or submitted to the MHRA.'The MHRA guidance recommends that you should consider the following aspects of RWD: 'Is the time between the occurrence of events and availability of the data to the study team suitable for the usage of the data in the study?How soon will data be able to be analysed after last patient, last visit?If used for monitoring adverse events, what impact would the availability of data have on the suitability of the database?' In relation to these considerations, are there situations where you have discounted/would discount using RCD?
The MHRA guidance states that 'Processes should exist for the resolution of discrepancies and communication of issues identified by later processing'.

Yes
No Would a common SOP for resolution of discrepancies for data providers be helpful?

Topic 3 -Internal pilot to look at validity of routine data for trial outcomes
It can be challenging to access relevant RCD in advance of a study being funded.There are advantages and disadvantages of receiving RCD partway through the trial vs at the end of the trial.Considerations include: the cost to download data periodically; the amount of work needed for every download and its management; the benefits from understanding the data structure, content, quality prior to final analysis.

Yes
No Are you aware of trials using RCD for outcomes where an internal pilot would be/would have been helpful?Which practical/logistical aspects of using routine data would be most helpful to publish, e.g.time to receive data, cost, processing challenges/guidance etc?For example: https://trialsjournal.biomedcentral.com/articles/10.1186/s13063-017-2135-9 What decision criteria would you use to decide between using RCD for trial outcomes vs using more traditional ways of collecting data for trial outcomes vs RCD being supplemented by more traditional data collection methods?

Workshop attendance
Are there other important considerations to inform the decision to use RCD for trial outcomes, which have not been described above?Do you have a case study related to any of the topics above that you would be willing to share here or at the meeting in March?This may be where you are currently assessing feasibility of using routine data for outcomes, or it may be a trial which is ongoing but where you have learnt some lessons to share.Please provide some details.
Please provide any references you are aware of that discuss any of the topics above.
Would you like to be considered for a place at the workshop to develop the guidance on Tuesday 7 March, 10am-4pm (approx.) in London?If so, please provide: Your email address Evidence from a previous feasibility study Evidence of previous use in a research study Expert opinion from someone familiar with the data source Algorithms to derive outcomes published Other -please specify What evidence would you consider sufficient to establish the reliability/validity of RCD for trial outcomes?(tick as many as appropriate) If you selected Other, please specify: Topic 2 -Timeliness of data capture for target outcomes and frequency (actual and expected) of data receipt If you selected Yes, please specify: What Stop/Go/Amend progression criteria might you consider in such an internal pilot?Topic 5 -Lack of published detail about practical issues related to use of RCD for trial outcomes Whether raw data or an analysis-ready dataset will be provided Onward data sharing ability, e.g. for a safety monitoring report Level of anonymisation Other Topic 6 -Decision-making